DanfeNER - Named Entity Recognition in Nepali Tweets

Twitter allows users to easily post tweets on any subject or event anytime, generating massive amounts of rich text content diverse topics. Automated methods such as Named Entity Recognition (NER) are required process the tweet data. Processing tweets, however, poses a special challenge they informal posts with incomplete context and often contain acronyms, hashtags, misspellings, abbreviations, URLs due length constraints. This paper presents first systematic study NER in Nepali corresponding five different entity types: Person Name (PER), Location (LOC), Organization (ORG), Date (DAT), Event (EVT). We develop DanfeNER, human-labeled high-quality benchmark data sets for low-resource language Nepali. DanfeNER contains 5,366 records 3,463 entities its train set 2,301 1,503 test set. Using this set, we several state-of-the-art monolingual multilingual transformer models, obtaining micro-averaged F1 scores up 81%.